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Approximations  to  the  Cumulative  Distribution  Function  of 
The  Magnitude-Squared  Coherence  Estimate 

Introduction* 

The  cumulative  distribution  function  of  the  estimate  of  magnitude-squared 
coherence,  obtained  by  averaging  over  N  statistically  independent  pieces  of 
Gaussian  data,  is  available  in  reference  1  as  a  sum  of  N-l  hypergeometric  functions. 
Direct  evaluation  of  this  quantity  has  recently  been  simplified  in  reference  2; 
however,  it  is  still  a  tedious  and  time-consuming  calculation  for  large  N  and  suffers 
from  overflow  and  underflow  unless  special  care  is  taken  in  programming.  Fur¬ 
thermore,  no  simple  result  for  obtaining  confidence  limits  is  available. 

In  this  report,  we  investigate  the  suggestion  of  Fisher  (reference  3)  that  the 
nonlinearity  arc  tanh  (v/x* )  converts  the  magnitude-squared  coherence  estimate  to  a 
ncar-Gaussian  random  variable.  In  particular,  we  present  simple  approximations 
for  the  mean  and  variance  of  this  nonlinearly  distorted  magnitude-squared 
coherence  estimate  and  fit  a  Gaussian  cumulative  distribution  function  over  a  wide 
range  of:  N,  the  number  of  pieces  averaged  in  the  magnitude-squared  coherence 
estimate;  C,  the  true  magnitude-squared  coherence;  and  P,  the  values  of  the 
cumulative  distribution  function.  Inversion  of  the  Gaussian  cumulative  distribution 
function  affords  a  simple  way  of  getting  confidence  limits  for  specified  probabilities 
of  threshold  crossings . 

Since  the  arc  tanh  (Vx~)  nonlinear  distortion  takes  no  account  of  the  known 
number,  N,  of  pieces  entering  the  magnitude-squared  coherence  estimate,  an  im¬ 
proved  nonlinear  distortion  that  utilizes  this  information  is  presented;  it  converts 
the  magnitude-squared  coherence  estimates  to  more  nearly  a  Gaussian  random 
variable  over  a  wider  range  of  parameters  N,  C,  and  P.  Evaluation  of  confidence 
limits  requires  the  inversion  of  this  nonlinearity;  analytic  inversion  is  not  possible, 
but  a  numerical  procedure  converges  rapidly. 


Investigation  of  Arc  tanh  (s/iT )  Nonlinearity 


Moments 

The  probability  density  function  of  the  magnitude-squared  coherence  estimate, 
C,  as  given  in  reference  l,is 

Pi(x)  =  (N-l)  (1~C)-(-~x^N~2  F(l-N,  1— N;  1;  Cx)  for  0 <  x <  I  and  N  >  2,  (1) 

(1-CX)2N‘ 

where  N  is  the  number  of  pieces  averaged;  C  is  the  true  magnitude-squared 
coherence;  and  F  is  a  Gaussian  hypergeometric  function. 


*  Portions  of  this  work,  which  were  done  jointly  with  G.  C.  Carter  (NUSC),  are  being 
prepared  for  journal  publication. 
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The  nonlinearly  distorted  version  of  random  variable  5, which  we  are  interested 
in,  is 


D  =  arc  tanh  (\/P) 


_1_  1  +  VC\ 

-  2 


(2) 


If  the  nonlinear  operation  in  (2)  results  in  a  Gaussian  random  variable  for  D,  then 
we  will  be  interested  in  the  mean  and  variance  of  D.  This  problem  is  considered 
analytically  in  appendix  A;  the  only  closed-form  results  that  we  can  obtain  are  listed 
below.  Let  the  m-th  moment  of  D  be  denoted  by 

fim(N,C)  =  D*  =  f  dx  I  arc  tanh  (v/x~ )  )m  p,(x) ,  (3) 

•'o 


where  the  overbar  denotes  a  statistical  average.  Then  for  C  -  0,  the  mean  of  D  is 


Mi(N.O)  = 


\/n  f(N-l) 

2  f(N-l/2) 


~  y/^/2 

VW 05 


as  ; 


(4) 


the  m-th  moment  is 


(5) 


and  the  variance  is 

o^N.O)  =  m2(N,0)  -  (N.O)  as  N^°°  • 


(6) 


This  last  expression  is  the  asymptotic  variance  of  D  for  C  =  0. 

In  order  to  deduce  the  fundamental  behavior  of  the  mean  and  variance  of  D  for 
C  *  0,  it  was  necessary  to  evaluate  (3)  numerically;  this  problem  is  considered  in 
appendix  B.  The  main  results  of  the  numerical  investigation  are  listed  below.  We 
find  mean 


M,(N,C)  a  arc  tanh  [\TCTB )  for  C  >  0  ,  (7) 


where 


B  = 


1-C2  t 
2(N-1)  ’ 


(8) 


and  variance 


oH  N.C)  a 


I 

2(N-1) 


for  C  >  0  . 


(9) 


2 


The  imprecise  qualifier  C  >  0  reflects  the  fact  that  the  probabilistic  behavior  of  C 
and  D  is  distinctly  different  for  C  =  0  versus  C  >  0;  for  example,  compare  (6)  and 
(9).  The  precise  region  where  (7),  (g),  and  (9)  are  valid  will  become  clear  in  later 
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plots.  The  result  (9)  is  a  slight  modification  of  Fisher  that  better  fits  the  calculated 
values  of  variance  in  appendix  B  and  the  slope  of  the  cumulative  distribution 
function  plots.  Its  independence  of  true  magnitude-squared  coherence  value  C  is 
striking  and  convenient. 


Probabilistic  Statements 

If  D  is  nearly  a  Gaussian  random  variable  with  mean  m,(N,C)  and  variance 
o^N.C),  then  the  cumulative  distribution  function  of  D,  i.e.,  the  probability  that  D 
is  less  than  some  threshold  T,  is  given  by  the  approximation 

P2(T).Prob(D<T)«£  7?^  «P  (-  00) 

where 

««  -  £  exp<-u!/2)  oo 

is  the  cumulative  distribution  function  for  a  zero-mean  unit-variance  Gaussian 
random  variable.  A  plot  of  (10)  on  normal  probability  paper,  with  T  as  the  linear 
abscissa,  yields  a  straight  line  with  abscissa-intercept  p,  and  slope  l/o,  since  the 
nonlinear  transformation  to  plot  ordinates  in  this  case  is  the  inverse  function*  4>*(  ). 

The  exact  cumulative  distribution  function  of  D  is  given  by 

P2(T)  =  Prob  {D<  T}  =  Prob  (arc  tanh  V£<T} 

=  Prob  {  C<  tanh^T)}  =  P,(tanh2(T))  ,  (12) 

where  we  employed  (1)  and  (2)  and  defined  P,  as  the  cumulative  distribution  func¬ 
tion  of  C: 

P,(A)  =  Prob{C<  A}  =  J  dx p,(x)  .  (13) 

Jo 

Thus,  we  can  relate  cumulative  distribution  function  P2  to  the  known  cumulative 
distribution  function  P,  given  in  reference  1.  Programs  for  the  exact  evaluation  of 
(13)  and  (12)  are  given  in  appendices  C  and  D,  respectively. 

The  availability  of  approximation  (10)  for  the  cumulative  distribution  function  of 
D  enables  us  to  give  a  simple  approximation  to  the  cumulative  distribution  function 
P,  of  C;  namely 

P,(A)  =  Prob  {C  <  A}  =  Prob  {D  <  arc  tanh  WK) } 

=  P2  (arc  tanh  (\[K ))  a  »  ^-rc  — ^  ^  ■  (U) 

where  Hi  and  o  are  given  by  (7),  (8),  and  (9). 

•A  superscript  I  ( 1 )  will  be  used  to  denote  the  inverse  of  a  function. 
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Furthermore,  we  can  now  solve  the  inverse  problem  of  determining  a  threshold  A 
for  specified  values  of  cumulative  distribution  function  P,  and  for  given  parameter 
values  C  and  N.  From  (14),  we  have 


A  *  tanh2  0it  +  o  4>*(P|) )  , 

where  p,  and  a  are  in  (7),  (8),  and  (9).  Simplification  of  (1 5)  results  in 


A* 


(— T- 

\i  +  af)  / 


where 


-[-sSj 


1/2 


0  =  tanh 


'  JL'Cfi)  1 . 
\/2(N-l)J 


(15) 


(16) 


(17) 

(18) 


A  program  for  the  evaluation  of  (16),  (17),  and  (18)  is  given  in  appendix  E. 

Plots 

Before  embarking  on  the  plots  of  exact  cumulative  distribution  function  P2  of 
distorted  random  variable  D,  as  given  by  (12),  we  plot  the  exact  cumulative 
distribution  function  P,  in  (13)  of  the  original  random  variable  C,  as  given  by 
reference  1.  Plots  on  normal  probability  paper  for  N  =  8,  16,  32,  64,  128  are  given 
in  figures  1-5*,  respectively.  In  these  figures,  each  dotted  straight  line  corresponds 
to  a  Gaussian  random  variable  with  mean  and  variance  as  derived  for  magnitude- 
squared  coherence  estimate  £  in  reference  4,  page  20.  The  discrepancy  with  the 
exact  cumulative  distribution  function  (in  solid  lines)  indicates  that  C  is  not  well- 
approximated  by  a  Gaussian  random  variable,  especially  for  the  small  and  large 
values  of  C  and  the  extreme  values  of  probability  near  .01  and  .99. 

Plots  of  the  exact  cumulative  distribution  function  P2  of  D,  given  by  (12),  are 
presented  in  figures  6-10  for  N  =  8,  16,  32,  64,  128.  Dotted  straight  lines 
corresponding  to  a  Gaussian  random  variable  satisfying  (7)-(10)  have  been  drawn 
for  every  C  and  N  value  being  considered;  however,  they  have  been  overdrawn  by 
the  exact  cumulative  distribution  function  (12)  in  some  cases  and  are  not  visible. 
The  agreement  between  exact  and  Gaussian  cumulative  distribution  functions  is 
extremely  good  except  for  very  low  values  of  C  and  N.  Notice  also  that  the  curves 
are  approximately  parallel  straight  lines  over  a  wide  range  and  can,  therefore,  be 
interpolated  more  easily.  Thus,  approximate  probabilistic  relation  (10)  and  its 
inverse  (16)  are  very  useful  and  accurate  for  a  wide  range  of  C  and  N,  encompassing 
most  of  the  useful  values  of  these  parameters. 


*  All  figures  are  grouped  at  the  end  of  the  main  text. 
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An  Improved  Nonlinear  Transformation 
Desired  Transformation 

The  arc  tanh  (0T )  nonlinear  transformation  in  (2)  is  independent  of  the  known 
quantity  N,  the  number  of  pieces  used  in  getting  magnitude-squared  coherence 
estimate  C.  An  improved  transformation  that  utilizes  N  should,  therefore,  be 
possible  and  should  be  investigated  to  see  if  a  more-nearly  Gaussian  random 
variable  can  result. 

The  method  to  determine  this  nonlinear  transformation  is  as  follows:  suppose  a 
random  variable  with  cumulative  distribution  function  P,(X)  is  given,  and  a 
nonlinear  monotonically  increasing  transformation, 

y  =  g(x) .  (19) 

that  will  result  in  a  specified  cumulative  distribution  function  P2(Y\is  desired.  We 
have 


P,(X)  «  Prob{x  <  X}  =  Prob{g(x)<g(X)} 

•  Prob{y<g(X)}  =  P2(g(X))  .  (20) 

Therefore,  inverting  this  equation,  the  desired  nonlinearity  is 

g(x)  =  P'(P,(x)).  (21) 

Thus,  we  need  to  know  the  inverse  of  the  desired  cumulative  distribution  function 

P2- 

In  particular,  if  y  is  Gaussian  with  mean  p,  and  variance  o2,  then 

P2(Y) . 

P»(t)  =  Ml +  (*>»(<),  (22) 

yielding 

g(x)  =  m,  +  od»'(P,(x))  .  (23) 

This  nonlinear  transformation  (23)  will  result  in  a  Gaussian  random  variable  with 
mean  ^and  o2.  However,  the  quantities  p,and  P,  depend  on  true  coherence  C  as 
well  as  N;  see  (7)  and  (1).  Since  C  is  unknown  (we  are  using  £  to  estimate  C),  (23)  is 
useless  unless  it  turns  out  that  (23)  is  substantially  independent  of  C;  it  can,  of 
course,  depend  on  N.  To  ascertain  this  possibility,  we  have  plotted,  in  figures  11, 
12,  and  13,  the  nonlinear  distortion  (23)  for  N  =  8,  16,  and  32,  respectively;  the 
program  is  given  in  appendix  F.  Within  each  figure  corresponding  to  a  fixed  value 
of  N,  the  five  distortions  (23)  for  C  =  .01,  .05,  .1,  .5,  and  .99  have  been  plotted  as 
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solid  tines;  in  addition,  the  arc  tanh  (\/x~ )  nonlinearity  is  superimposed  in  dotted 
lines.  Furthermore,  (23)  has  only  been  plotted  in  the  range  of  probabilities  (.01,  .99] 
for  P,,  since  this  is  the  range  we  are  concentrating  on. 

The  ideal  situation  would  occur  if,  within  each  figure,  the  same  identical 
nonlinear  function  occurred  for  all  five  cases  of  C  =  .01,  .03,  .1,  .3,  and  .99;  of 
course,  the  function  can  change  with  N.  Although  this  fortuitous  situation  does  not 
occur  precisely,  inspection  of  each  of  the  five  cases  within  each  figure  reveals  a 
marked  independence  of  C.  Thus,  the  desired  goal  of  converting  magnitude-squared 
coherence  estimate  C  to  a  nearly  Gaussian  random  variable,  by  means  of  a 
nonlinear  transformation  that  depends  only  on  N,  is  achievable. 

Now,  the  problem  remains  to  find  as  simple  an  analytic  transformation  as 
possible  to  approximate  the  solid  curves  in  figures  11,  12,  and  13.  A  great  deal  of 
trial-and-error  has  resulted  in  the  following  candidate.  The  distorted  version  of  C 
we  adopt  is  not  (2),  but  rather 

Dn  =  arc  tanh  (gN(C))  ,  (24) 


where 


(l-x)N  In 


gN(x)  =  \/x  + 


(t). 


2(N  +  2) 


;  g«(x)  =  0T. 


(25) 


The  nonlinear  transformatioh  in  (24)  and  (25)  depends  on  N  as  well  as  x,  but  is 
independent  of  the  unknown  parameter  C.  The  leading  term  of  (25)  is,  of  course, 
the  earlier  result.  The  correction  term  goes  quickly  to  zero  as  x-*l,  and  as  N  in¬ 
creases,  as  figures  1 1, 12,  and  13  indicated  desirable. 


Probabilistic  Statements 

In  order  to  measure  the  success  of  transformation  (24)  and  (25),  we  need  to 
evaluate  the  exact  cumulative  distribution  function  of  random  variable  DN.  We 
have  the  exact  relation 

Prob{DN<T}  =  Prob{arctanh(gN(C))<  T} 

=  Prob{gN(C)<tanh(T)} 

=  Prob{C<  gj5j(tanh(T))}  =  P,  (gJ,(tanh(T) )  )  ,  (26) 

where  gj,  is  the  inverse  function  of  gN,  and  P,  is  the  cumulative  distribution  function 
of  £.  The  program  for  evaluation  of  (26)  is  given  in  appendix  G. 

If  we  have  succeeded  in  realizing  a  Gaussian  random  variable  for  DN,  then  an 
approximation  to  (26)  is  given  by 


Prob{DN<  T} 


(27) 
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where  we  have  chosen  to  use  (7),  (8),  and  (9)  here  also.  Therefore,  an  approximation 
to  cumulative  distribution  function  P,  follows  according  to 

P|(A)  =  Prob{C<  A} 

=  Prob{arc  tanh(gN(C))  <  arc  tanh(gN(A»} 


=  Prob{  Dn  <  arc  tanh(gN(A))} 


a* 


( 


arc  tanh(gN(A»  - 


*) 


(28) 


Furthermore,  (28)  affords  an  approximate  expression  for  the  required  threshold 
value,  A,  to  use  for  a  specified  probability  value  P,;  it  is 


Aag},  (tanhOi,  +  P,)  )  ) 


(29) 


This  relation  generalizes  (15),  which  applied  to  the  arc  tanh  (\^T  )  transformation. 
The  main  drawback  to  (26)  and  (29)  is  that  we  need  the  inverse  function  to  (25). 
(This  inverse  is  not  needed  for  (27)  or  (28).)  When  the  correction  term  in  (25)  is  zero 
(as  for  N  =  «»),  the  inverse  relation  is  simply  g£,(y)  =  y2;  however,  no  such  simple 
relation  exists  for  (25)  in  general,  and  gjj,  must  be  found  numerically.  It  would  be 
worthwhile  to  uncover  a  simpler  nonlinearity  than  (25)  that  still  converts  C  to  a 
nearly  Gaussian  random  variable,  and,  at  the  same  time,  is  easily  invertible; 
whether  this  is  possible  is  unknown. 


Plots 


The  final  measure  of  success  of  transformation  (24)  and  (25)  is  afforded  by 
figures  14-18  for  the  cumulative  distribution  function  of  DN  for  N  =  8,  16,  32,  64, 
and  128,  respectively.  The  discrepancy  from  a  Gaussian  random  variable 
(dotted  lines)  is  far  less  than  for  the  corresponding  cases  in  figures  6-10  for  the 
arc  tanh  (\/x  )  nonlinearity.  Even  for  as  low  a  value  as  N  =  8,  a  fair  approximation 
is  given  forC  =  .01. 


Summary 

Two  nonlinear  transformations  have  been  presented  that  convert  magnitude- 
squared  coherence  estimate  C  to  nearly  a  Gaussian  random  variable.  They  afford 
quick  approximate  evaluation  of  the  cumulative  distribution  function  of  C  and  can 
be  used  for  arbitrarily  large  N;  see  (14)  and  (28).  They  also  allow  for  rapid 
calculation  of  confidence  limits  since  these  relations  can  be  inverted  to  find  the 
threshold  required  for  a  specified  cumulative  distribution  function  value;  see  (15) 
and  (29).  A  drawback  of  the  latter  more-accurate  result  is  that  no  simple  analytical 
inversion  to  nonlinear  transformation  (25)  exists;  however,  since  computer  aid 
would  likely  be  used  to  evaluate  the  expressions  given  here,  inversion  of  (25)  is  easily 
achieved.  A  simple  recursive  scheme  was  employed  by  the  author;  see  appendix  G. 
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11a.  C  -  0.01 


Figure  11.  Deeired  Non-Linearity  for  N  -  0 
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lid.  C  *  0.5 

Figure  11.  (Cont'd)  Desired  Non-Linearity  for  N  =  8 
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0  .1  .2  .3  .4  .5  .6  .7  .8  .9  1 

X 

lie.  C  -  0.99 

Figure  11.  (Cont’d)  Desired  Non-Linearity  for  N  ■  8 


0  .1  .2  .3  .4  .5  .6  .7  .8  .9  1 

X 
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Figure  12.  Desired  Non-Linearity  for  N  -  16 
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12b.  C  =  0.05 
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Figure  12.  (Con(’d)  Desired  Non- Linearity  for  N  *  16 
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Figure  17.  Cumulative  Distribution  Function  of  DN  for  N  -  64 


Figure  18.  Cumulative  Distribution  Function  of  DN  for  N  =  128 
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Appendix  A 

Moments  of  D  for  C  =  0 


Exact  Results 


For  C  =  0,  the  probability  density  function  in  (1),  of  magnitude-squared 
coherence  estimate  £,  simplifies  to 


(A-l) 


p,  M  *  (KMX!-  vj*  -for  0<**l  a*J  N*  2. 

The  m-th  moment  (3)  is  then  given  by 

(Nj,o)  *  JQd*  f,(*)  Orz  (A.2) 


Let 


to  obtain 


x  -  Wit),  Jx  .  Jt  2  j.kltyc.k’IH,  «A-3) 


(A-4) 


For  m  =  0,  the  substitution  u  =  cosh(t)  immediately  yields  unit  area.  So,  let 
m  >  1  in  the  following.  Integrate  by  parts  with 


j"  v  a  siwViR) 
U  *  t  ^  qV  =  at  ilnl-r 


cwirw  ' 


and  obtain 


m*.(M 


[■ 


w-  I 

=  rh  \  dt  — Tiw-ioT  /or  Nil. 


-0  CwTMt) 

We  now  specialize  to  m  =  1 ;  let  x  =  e1,  dx  =  x  dt,  p  =  2N-2,  and  obtain 

MM  •  2^*  jC  -  2*~ -£*4 

'  Jo  (i+  x/  r(2N-^ 

=/>.(«".•) Frr Ni3'^-o)*1, 


(A-5) 


(A-6) 


(A-7) 
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26 


CO»M*)  m  eS  t*  I"  *  -  OK  walt(e^), 


(A- 13) 


when  we  used  reference  6, 4.6.38.  Then 

fimitijo)  -  mf*  CWCtee'1*,  (A- 14) 

where 


•f*  (t)  *  arc  cosli  (e*),  p*2N“2. 

(A-15) 


We  are  now  in  a  position  to  employ  Watson’s  Lemma,  reference  7,  pp.  102- 
103.  We  need  the  expansions  of  f,(t)  and  f2(t)  about  t  =  0.  Consider 

riL)  -  1  *■  *  +  OK-*)  .  _L  i  +  *  +  OK*) 

'  {I^WTokJ  &  i+it+or) 


*jtr[,++t  +  o^ 


(A- 16) 


Also  via  reference  6, 4.6.21  and  4.1 .24, 

(,  (ft  -  u  [**  +  £*1?] 

*  In  fl+  t  +■  0  It*)  +  {X+U'+Vl iFfj 

=  l»[i+i  +  oie)  +  /I?{n-ti  +otr)f  J 
*&*+■  or)] 

.  & + t  +£t*+  OK')'  i[li  *  i\F{*+  Ok')}  -t-orj ) 

=  lA-17» 


27 


TR6327 


Therefore, 


O)-  (2+)V(H^t  +  0/f*)], 


(A- 18) 


and  using  (A- 1 6) 


f,(0O) - fetf  "'[i+  +oK-*)j  w 


Term-by-term  integration  of  (A-19)  in  integral  (A-14)  yields  the  desired  result 

~  ■ M  N-* -- 

(n-^T 


(A-20) 


Special  cases  are 


sR/i 

fh-'-2S 


as  hi  -*  00  i 


(A-21) 


+ “  w^~"  (A-22) 

r'  =  ~ 

~  (l- ?)(*>- -£$'  -  w— • 


(A-23) 


Equation  (A-21)  agrees  precisely  with  (A-8)  when  the  latter  is  expanded  to  two  terms 
in  (N-l)'1.  More  generally,  (A-20)  extends  (A-10)  to  two  terms  in  the  asymptotic 
expansion,  and  (A-23)  generalizes  (A- 11)  to  two  terms.  Comparison  of  (A-20MA-23) 
with  results  of  the  numerical  procedure  in  appendix  B  reveals  very  close  agreement. 
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Third  Cumulant  of  Normalized  D  for  C  -  0 


In  order  to  determine  whether  random  variable  D,  for  C  =  0,  is  approaching  a 
Gaussian  random  variable  as  N  increases,  we  evaluate  the  third  cumulant  of  the 
normalized  random  variable 


r  - 

<r 


We  have  the  usual  first  two  cumuiants  of  r: 


(A-24) 


(A-25) 


Then,  using  (A-20MA-23), 


7T  -  »  - 


X1?  =  (r-  rf  *  r*  = 
g  blltihlld  ^  =.63)  «  W-- 


(A-26) 


Since  this  quantity  does  not  decrease  to  zero  as  N  increases  to  «®,  the  nonlinear 
distortion  D  of  magnitude-squared  coherence  estimate  C  does  not  approach 
Gaussian,  at  least  in  the  case  when  C  =  0. 


Recursion  for  pm(N,0) 
From  (A-l)  and  (A-2), 


*  J0*  fifi) , 

(A-27) 

where 

A  “  Qrvfemli  (f)  >  A  llrl  -  |  j  ■ 

(A-28) 

Lett  =  V5T  in  (A-27)  to  obtain 

2i(N-l)0-  V-T’^dr). 

(A-29) 

Let  m  >  1  in  the  following  and  integrate  by  parts  with  u  =  Am(t),  v  = 
obtain 

N  (n,o)  =  *  Jo  df  (i  -  1 

-G-t2)N-',to 

(A-30) 

29 


Now  we  assume  that  N  >  3  and  develop 


/*«(  N,o)  -  M  It) 

-  r»  (n-  i,»)  - « J>  (i  -  tfvrw  (A.31) 

Integrate  by  parts  with  u  =  tAm''(t),  v  =  -( 1  -t2)N_2/(2(N-2)J ,  to  obtain 

/*-M  0)  -  M  -  (l-tflT* /Tfc).  (A-32) 

For  m  =  1,  this  immediately  reduces  to  the  recursion  in  (A-7).  Furthermore, 
for  m  =  2,  the  last  integral  can  be  evaluated  immediately,  and  we  find  the  con¬ 
venient  recursion  for  the  second  moment: 


^  h>3,^m-2U2. 

(A-33) 

The  starting  value  for  ^i2(2,0)  follows  from  (A-30)  and  (A-28): 


=  2  In  2. 

(A-34) 


We  now  assume  that  m  >  3  and  conclude  the  recursion  derivation  in  (A-32). 
Integrate  by  parts  with  u  =  Am'2(t),  v  =  -( 1  -t2)N_2/[2(N-2)],  to  obtain  the  final 
result 

l>»M  ■  UfrAtN-'W  -  firN»3 

(A-35) 

(Equation  (A-35)  actually  holds  for  m  =  2  also;  in  fact,  it  reduces  to  (A-33).) 

In  order  to  start  this  recursion,  we  need  to  know 

2  O'  1  7  (A-36) 

We  already  know  |a2(2,0)  in  (A-34),  so  keep  m  >  3  in  the  following.  Let 
x  *  (l-t)/(l +t)  and  obtain 


Integrate  by  parts  with  u  =  (-In  x)"1'1,  v  =  x/(l  +  x),  to  obtain 


Now  let  x  =  e'1  and  get 


The  value  for  *i4( 2,0)  requires  knowledge  of  {(3);  values  of  {(n)  for  n  >  2  are  given  in 
reference  6,  page  811.  Thus,  we  now  have  all  the  starting  values  necessary  to  use 
recursion  (A-35)  Form  >3. 
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Appendix  B 

Numerical  Evaluation  of  Moments  of  D 


Our  starting  point  is  the  top  line  of  (A-4),  modified  to  the  case  for  C  #  0.  We  then 
express  the  hyperbolic  functions  in  terms  of  exponentials  and  obtain 


where 


E  -  e*r(-2t), 


(B-2) 


and  p,  is  given  by  (1).  A  program  for  evaluation  of  (B-l)  is  given  below.  Inputs  are 
N,  C,  and  M(  =  m)  in  the  first  three  lines.  This  program  was  used  to  deduce  the 
behaviors  listed  in  (7),  (8),  and  (9). 


to 

N  =  128 

20 

C= .  5 

30 

M=1 

40 

Sl-O 

30 

S2*8.'SQR<N:m-16''N-M 

60 

COM  N,C,M 

70 

F*<N-1 )#<  t-C >  'N 

80 

PRINT  "N  *  "  ;  N , " C 

*";c,  "i 

30 

S3*<FNS<S1 >+FNS< 

$2 >  >* .  5 

too 

34*0 

no 

S5*2 

120 

36=<S2-S1  >* . 5 

130 

S?*2-'3 

140 

FOR  38*1  TO  35-1 

STEP  2 

130 

S4*S4+FNS<S1+S6*S8> 

160 

NEXT  S8 

170 

0*<S3+2*S4  '*36*3 

7*F 

180 

PRINT  USING  "M.  12DE,  78’1 

190 

33*33+34 

200 

S4-0 

210 

S5*S5«2 

220 

S6*S6* . 5 

230 

GOTO  140 

TR  6327 


240  DEF  FNS<  T > 

250  COM  N,C,M 

260  £»EXP<-2*T) 

270  D*l-E 

280  S«1*E 

290  ft«<D/SX2 

300  RETURN  3*£*D''S"3*FNP<fl)*T'M 
310  FNEND 

320  DEF  FNF21<fl,B,C,X) 

330  F«F4»1 

340  FOR  F5-1  TO  100 

350  F6«F5-1 

360  F4»F4*<fl*F6>*<B»F6)*X/'<(C+F6)*F5) 

370  F»F*F4 

380  IF  RBS<F4XlE-12*flBStF>  THEN  420 
390  NEXT  F5 

400  DISP  “100  TERMS  RT  “;fl{B;C;X 

410  PRUSE 

420  RETURN  F 

430  FNEND 

440  DEF  FMP<X> 

450  COM  N,C,M 

460  B=C  *X 

470  P»<l-X>  '<N-2>-  <1-BX<2*N-1>#FNF21<1  -N, i-N, 1 .  B> 

480  RETURN  P 

490  FNEND 
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Appendix  C 

Program  for  the  Evaluation  of  (13) 


10  N=16 

20  OUTPUT  0;  "N  =“;N 

30  PLOTTER  IS  "GRAPHICS" 

40  GRAPHICS 

50  DATA  .01, .025, .05, . 1, .2, .3, .4, .5, .6, .7, .8, .9, .95, .975, .99 
GO  DIM  GUMS)  *  GRID  LINES 

70  READ  GO) 

80  X»»l 

98  Y»«FNlriMpM  15>  ) 

1O0  SCALE  0,  Xi»,  -Ym,  Yih 

llO  FOR  X*0  TO  Xm  STEP  X».'10 

120  MOVE  X , Ym 

130  DRAW  X, -Ym 

140  NEXT  X 

150  FOR  1-1  TO  15 

180  Y«FN!r.vpM  (CU)) 

170  MOVE  0 , Y 

180  DRAW  Xm.Y 

190  NEXT  I 

200  PENUP 

210  DATA  .01, .05, . 1 , .2, . 3, .4, .5, .6, . 7, .8, . 9. . 95, .99 
220  DIM  COM3)  '  TRUE  COHERENCE  VALUES 

230  READ  CO) 

240  FOR  I«3  TO  11 

250  C-CU) 

260  T«<l-C>A2 

270  Bin««T*U*2*C  H  > -N 

280  Sigftta*(N-l )«T*v2*C*'  1 -6*C+ 1 3*C*t  >- N ) /' H*< N+ 1 > > 

290  Sigm**SQR($igM«) 

300  LINE  TYPE  3 

310  FOR  Td-0  TO  X».  STEP  X*MO0 

320  Y«< Td-C’lt»» •  'Sigma 

330  PLOT  Td,Y 

240  NEXT  To 

350  PENUP 

360  LINE  TYPE  1 

370  FOR  Td«0  TO  Xm  STEP  X*.- 100 

380  Tc«Td 

390  Y“FNCdfw»c  <N, C , Tc  ) 

4O0  PLOT  Td,  FNlrv<phi  vY> 

410  IF  V  .99  THEN  438 

420  NEXT  Td 

438  PENUP 

440  NEXT  I 

450  NEXT  K 

468  END 
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DEF  FNCdfr»sc(N,C,Ct  ) 

0“  1  -Ct. 

P»C*Ct 
H«l-P 
R«0  H 
Tr>*w«0 
S»T*  1 

FOR  K«1  TO  N-2 
Tol d*Tneu 
Tn«w«T 

T»<  <2*K-l-KN-K>*P>#Tn*w-<K-l  >*0*Told>*R.  K 

S*S+T 

NEXT  K 

P=Ct  *  <  >:  1  -C )  H )  i'N#S 

RETURN  P 

FNEND 

DEF  FN I  rwph  i  <  X  ) 

IF  <X>-0>  AND  <X<  *1  )  THEN  676 

PRINT  “ARGUMENT  ";X;"IS  DISALLOWED  FOP  I  EROE  PHI  FUNCTION" 

STOP 

IF  <X>0>  AND  <X<1)  THEN  700 
P»9.99999999999E99*<2*X-I> 

GOTO  750 
p*>: 

IF  X > . 5  THEN  P«.5-<X-.5> 

P“SQR ( -2*L0G<P> I 

P=P-<2. 51551 7+P*<.802853«-P*.0 10328  1  »P*<  1. 432  7S6*P*<.  1 89269+P*.  00 1308)  > 

IF  X< . 5  THEN  P*-P 

RETURN  P 

FNEND 

DEF  FNT»nh(X> 

S-EXP<2*X) 

RETURN  <S-l  )/<$•*•  I  .) 

FNEND 

DEF  FNArc  t  *nh< X > 

RETUPN  .  5*L0G<  <  l*X>-'<  1  -X>  > 

FNEND 
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Appendix  D 

Program  for  the  Evaluation  of  (12) 


The  functions  Cdfmsc,  Invphi,  Tanh,  and  Arctanh  have  already  been  listed  in 
appendix  C. 


10  N*8 

28  OUTPUT  0;  "N  *“  ;N 

30  PLOTTER  IS  “GRAPHICS" 

40  GRAPHICS 

50  DATA  .  01 ,  .  025,  .  05,  .  1 ,  .  2, .  3,  .  4, .  5,  .  6,  .  7 .  .  8,  .  9.  .  $5,  .  *75,  .  99 

60  DIM  G< 1 1 15)  >  GRID  LINES 

70  READ  G<*> 

80  Xi»*4 

90  Y«**FNInvphi  <GC  15)  ) 

1 00  SCALE  0,  X«i,  -Ym,  Yta 

110  FOR  X*0  TO  Xm  STEP  X»'8 

120  MOVE  X, Y* 

130  DRAM  X,  -Y«i 

140  NEXT  X 

150  FOR  1*1  TO  15 

160  Y*FN Invphi <CC I > ) 

170  MOVE  0 , Y 

180  DRAM  Xm.Y 

190  NEXT  I 

200  PENUP 

210  DATA  .01, .05, . 1, .2, .3, .4, .5, .6, .7, .8, .9, .95, .99 
220  DIM  C < 1 : 1 3 >  !  TRUE  COHERENCE  VALUES 

230  READ  C<*> 

240  FOR  1*1  TO  13 

250  c*c<n 

260  B*<1-C''2)/<2*<N-1>> 

270  M**n*FNAre t *nh< SQR<C*B> ) 

280  S i gm** 1 -  SQR  <2*<N- 1 >  > 

298  LINE  TYPE  3 

3O0  FOR  Td*0  TO  Xm  STEP  Xm- 100 

310  Y*f Td-Me*n)  -Si gm* 

328  PLOT  Td, Y 

330  NEXT  Td 

340  PENUP 

350  LINE  TYPE  1 

36©  FOR  Td“0  TO  Xim  STEP  Xni'lDO 

370  Tc*FMTanh< Td>  2 

3S0  Y*FNCdf  t»*e  <  N,  C  ,  Tc  > 

390  PLOT  Td.FNInvphi ■ V  ■ 

400  IF  Y > . 99  THEN  420 

<10  NEXT  Td 

420  PENUP 

430  NEXT  I 

440  NEXT  K 

450  END 


37/38 
Reverse  Blank 


TR  6327 


Appendix  E 

Program  for  the  Evaluation  of  (16) 


The  functions  Invphi  and  Tanh  have  already  been  listed  in  appendix  C. 


16  INPUT  P, N, C  !  PROBABILITY, PIECES, COHERENCE 

26  ftlpha*SQR<C+<l-C*C>'*<2*<N-l  >>> 

30  T*FMInvphi  <  P  >'SGR<2*<N- 1  >> 

40  Be« »=FNT»nh<T> 

50  A»<  (Alph*'fB*t*>/'<  l+Alph»*B*t*>  >*2 

60  PRINT  a,p;n;C 

70  END 
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Appendix  F 

Program  for  the  Evaluation  of  (23) 


The  functions  Cdfmsc,  Invphi,  and  Arctanh  have  already  been  listed  in  appendix 
C. 


10  c*.i 

20  N*32 

36  PRINT  "C  ■";C,"N  «“;N 
4£»  1  -C*C> /(2#<N-1  )  ) 

50  M«a»i»FNArc  t  anh<.  SQR<  C+B)  > 

60  Si gma*l - SCR< 2*<N- 1 > > 

?0  PLOTTER  IS  '•GRAPHICS'* 

80  GRAPHICS 

90  SCALE  0, i,-.3,3.5 

188  GRID  . 1 , . 5 

110  PENUP 

120  LINE  TYPE  3 

130  FOR  X«0  TO  .995  STEP  .005 

140  Y«FNArctanh<SQR(X> >  !  ORIGINAL  NON-LINEARITY 

150  PLOT  X, Y 

160  NEXT  X 

170  PENUP 

180  LINE  TYPE  1 

190  FOR  X-.002  TO  .998  STEP  .002 
200  Pl*FNCdfi»*c<N,C,X> 

210  IF  P 1 < .01  THEN  250 

220  IF  P1>1-.01  THEN  260 

230  Y*Me an*S i g»a*FN Invph >  <F 1 >  !  DESIRED  NON-LINEARITY 

240  PLOT  X, Y 

250  NEXT  X 

260  END 
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Appendix  G 

Program  for  Evaluation  of  (26) 


The  functions  Cdfmsc,  Invphi,  Tanh,  and  Arctanh  have  already  been  listed  in 
appendix  C. 


to  It*  32 

20  OUTPUT  0;"N  =‘';N 

30  PLOTTER  IS  "GRAPHICS" 

40  GRAPHICS 

50  DATA  .01, . 025, . 05, .t,. 2, .3, .4, .5, .6, .7, .8, .9. .35. .975, .99 
£0  DIM  G < 1 : 1 5  >  !  GRID  LINES 

70  READ  G< * > 

SO  Xm*4 

90  Yi>i*FNIrwphi  <G<  15)  ) 

100  SCALE  0,  Km,  -Yin,  Ynt 

110  FOR  X-0  TO  Xm  STEP  Xi»/8 

128  MOVE  X,Ym 

130  DRAW  X,-Ym 

140  NEXT  X 

150  FOR  1*1  TO  15 

160  Y*FNIrwphi <C< I ) > 

170  MOVE  0,Y 

180  DRAW  Xm,Y 

190  NEXT  1 

200  PENUP 

210  DATA  .01, .05, . 1, .2, .3, .4, .5, .6, . 7, . 3, .9. . 95. . 9? 

220  DIMC<1!13>  !  TRUE  COHERENCE  VALUES 

230  READ  C<*> 

240  FOR  1*1  TO  13 

250  c*c<i> 

260  8*< 1 -C  2 > / < 2* < N- 1 )  ) 

270  M#*ri*FNArc  t  *nh<  S0R<  C+D  >  ) 

280  Si  giu*»  1  SQR<2*<  N-1  )> 

290  LINE  TYPE  3 

300  FOR  Td*©  TO  Xin  STEP  Xn.100 

310  Y*<Td-M«*n>'-Sigi»* 

320  PLOT  Td,  Y 

330  NEXT  Td 

340  PENUP 

350  LINE  TYPE  1 

360  FOR  Td*8  TO  Xiri  STEP  Xio- 100 

370  Tc*FNT*nh<Td> 

3C0  Tc*FNIriL-g(Tc,N>  1  DISTORTION  IS  gCN) 

390  Y»FNCdf mtc  <N,C, Tc  > 

400  PLOT  Td,  FNIiwph i  <  Y  •' 

410  IF  Y ? . 99  THEN  430 

420  NEXT  Td 

430  PENUP 

440  NEXT  I 

450  NEXT  I 

460  END 
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470  DEF  FNInvg<Y,N) 

460  X1-<Y>1.5/N)''2 

460  X2»X1*.01 

500  Fi=»3QR<Xl>*.5*a-Xl  W1#N>*L0G<.5*<N-1,»*X1><'<N«.2:>-Y 

510  F2*S0R<X2)  +  .5#O-X2>''<i*N)*L0CC.5*<N-l  ,  *X2>  -<M*2;-Y 

520  IF  RBS<F1-F2>< IE- 10  THEN  RETURN  X2 
530  T-<X1#F2-X2#F1>/'<F2-F1> 

540  X1-X2 

550  X2»T 
560  F1*F2 

57©  GOTO  510 
560  FNEND 
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